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Abstract Body 



Background/context: 

The increasing use of state-mandated public high school exit exams— tests each student 
must pass before he or she is awarded a high school diploma— is one manifestation of the 
current movement in U.S. public schooling toward more explicit standards of instruction and 
accountability. Unlike some aspects of accountability systems, the accountability consequences 
of failing an exit exam fall partly (or even primarily) on students, as opposed to the schools or 
districts. The number of states requiring students to pass an exam to graduate has increased 
from 18 in 2002 to 22 in 2007, with an additional four states intending to implement exit exams 
by 2015. Soon, over 70 percent of U.S. students will soon be subject to such exam 
requirements (see, e.g.. Center on Education Policy, 2004, 2005; Dee & Jacob, 2006; Warren, 
Jenkins, & Kulick, 2006). The effects of exit exam policies, however, remain somewhat unclear, 
despite a number of recent studies. Competing notions of how such exams might influence 
student and school behaviors lead to divergent predictions of how students will be affected. 
Some argue, for example, that a high school exit exam requirement will create incentives both 
for schools to provide better instruction to struggling students and for these students to work 
harder to learn more before graduation. On the other hand, others have argued that creating 
additional barriers to graduation discourages students— particularly academically and socially 
disadvantaged students— from persisting in school and hence leads to increased dropout rates 
and greater inequality (for discussion, see Dee & Jacob, 2006; Reardon & Galindo, 2002; Warren 
etal., 2006). 

Several recent papers have estimated the effect of failing an exit exam, given that an 
exit exam requirement is in place (Martorell, 2005; Papay, Murnane, & Wlllet, 2008; Reardon et 
al., 2008). Each of these papers uses a regression discontinuity design to estimate the effect of 
failing an exit exam, which yield valid causal estimates of the effect of failing for students near 
the margin of passing. Generally, each of the papers finds some evidence that failing an exit 
exam in 10 th grade increases the probability that a student will not graduate from high school, 
but because these papers rely on regression discontinuity, their estimates are informative only 
for students who are near the margin of passing. They do not tell us what the effect of failing 
the exam is for students with skill levels far below the passing score. 

Research that estimates the effect of exit exam policies (rather than the effect of failing 
an exam) has typically focused on estimating the average effect of the policy on all students, 
regardless of skill level, and has generally estimated the effect of a high school exit exam 
requirement on high school dropout or completion rates. The best of these studies find that 
high school dropout rates tend to increase by roughly 1 to 2 percentage points, on average, 
when states implement rigorous exit exams (Dee & Jacob, 2006; Warren et al., 2006). Dee and 
Jacob (2006) find that these effects are concentrated among black students and students in 
high-poverty schools. Nonetheless, none of the existing studies are able to provide estimates 
of the effects of high school exit exam policies on the students for whom they are likely to have 
the largest effects— students who enter high school with achievement levels in the bottom 
quartile of the income distribution 
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Purpose/objective/research question/focus of study: 

In this paper, we estimate the effect of a high school exit exam requirement (relative to 
no requirement) on students' academic achievement, persistence in high school, and 
graduation rates. We are particularly interested in the effects of the policy on the students 
who have low initial skill levels in high school. 



Setting: 

The study is based on data from four large California districts— Fresno, Long Beach, San 
Diego, and San Francisco Unified School Districts— to investigate the effects of failing the 
CAHSEE. These are four of the eight largest school districts in California, collectively enrolling 
over 110,000 new high school students (about 5.5 percent of high school students in the state) 
annually. We use three years of longitudinal data from students who were in 10 th grade in the 
Spring of 2003, 2004, 2005, 2006 (i.e., we use data from 2003-2008). 



Population/Participants/Subjects: 

The study draws on data from roughly 100,000 students from the four school districts. 
The students are roughly 25% White, 20% Asian, 40% Hispanic, and 15% Black. 



Intervention/Program/Practice: 

The California State Legislature passed Senate Bill SB2X in March 1999, requiring 
California local school districts to administer a high school exit exam (the California High School 
Exit Exam [CAHSEE]) and provide supplemental instruction to those students who do not 
demonstrate sufficient progress toward passing the exam. As implemented, the CAHSEE is a 
two-part exam of mathematics and English language arts (ELA) skills. The math section assesses 
students' mastery of the California math content standards for sixth and seventh grade and 
their Algebra I skills using a multiple-choice format. The ELA section is aligned with state 
content standards through grade ten and utilizes a multiple-choice format along with one 
essay. Both tests are administered in English, regardless of a student's primary language. 

The test is first administered to students in the Spring of 10 th grade, and students have 
at least five subsequent opportunities to retake the sections they have not yet passed (twice in 
11 th grade and 12 th grade, and at least once following the end of the 12 th grade school year). 
Testing dates are centrally scheduled by individual districts and the exam is administered over 
the course of two days (one day for each portion). The test is untimed, though students 
typically complete each section in three to four hours. Districts notify students and their 
parents of their CAHSEE performance about seven weeks after the exam is administered. 
Because students are told their exact score, not simply whether they passed or failed, students 
who fail have some sense of how close they came to scoring the requisite 350 they need to 
meet the CAHSEE requirement. 
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Research Design: 

We exploit a change in the implementation of the policy to estimate the effect of the 
exit exam requirement on student outcomes. In July of 2003, after the completion of the Spring 
2002-03 administrations of the CAHSEE (taken by 10 th graders in the high school class of 2005), 
the State Board of Education voted to defer the CAHSEE requirement for two years. As a result, 
students in the Class of 2005 took the CAHSEE exam in 10 th grade under the belief that passing 
would be required to graduate, but then were told several months later that they would not be 
subject to the policy. For those students who were in 10 th grade in Spring 2004 or later, 
however, the CAHSEE requirement has been in place since the beginning of their 10 th grade 
year. As a result, students in the class of 2005 and in later cohorts thought that they were 
subject to the exam through their 10 th grade years; the policy differed for them only after 10 th 
grade. 

In this paper, we compare the outcomes of students in the 2005 cohort with those from 
later cohorts, conditional on their standardized test scores in Spring of 10 th grade. The initial 
failure rate of the CAHSEE in 10 th grade in 2004 was roughly 25%. By comparing the results of 
students in the bottom quartile in 10 th grade in cohorts who were and were not subject to the 
CAHSEE requirement, we can estimate the effect of the requirement on their subsequent 
outcomes. Because this difference may be biased by other policy changes that affect the 
bottom quartile of students, we also estimate the between-cohort difference in outcomes for 
students in higher quartiles of the 10 th grade distribution. A pattern of between-cohort 
differences for bottom quartile students but not for higher quartile students would suggest 



Data Collection and Analysis: 

We use longitudinal student-level data from four large California districts— Fresno, Long 
Beach, San Diego, and San Francisco Unified School Districts— to investigate the effects of 
failing the CAHSEE. These are four of the eight largest school districts in California, collectively 
enrolling over 110,000 high school students (about 5.5 percent of high school students in the 
state) annually. For our primary analyses, we use data from four cohorts of students— defined 
as the cohorts scheduled to graduate in 2005, 2006, 2007, and 2008. For the first of these 
cohorts, the CAHSEE was not binding after 10 th grade; for the latter three cohorts, the CAHSEE 
was binding. 

We estimate the effect of the CAHSEE policy on three outcomes— academic 
achievement, persistence to 12 th grade, and graduation. We measure academic achievement 
using the Spring 11 th grade English Language Arts (ELA) California Standards Test (CST) score. 
We use the ELA rather than the Math CST score in 11 th grade because all students take the 
same 11 th grade ELA CST, but take one of a number of different subject/ content math CST tests 
(e.g.. Geometry, Algebra I), depending on what math course they are enrolled in. Thus, 11 th 
grade math scores are not comparable across students. 

Although we cannot directly determine whether students have dropped out of high 
school — because students who leave a given district prior to graduation may be dropouts or 
may have left and enrolled elsewhere— we can identify whether students are present in the 
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district two years after first taking the CAHSEE (in the Spring of 12 th grade). We use the 
indicator of presence in Spring of the scheduled 12 th grade year as an indicator of persistence in 
schooling. Of course, some students may not be present in the district because they have 
transferred to another district. Nonetheless, if we observe that failing the CAHSEE affects the 
probability that a student is present in the district in 12 th grade, we can assume that this is 
because failing the CAHSEE affects persistence/dropout rates. It is unlikely that CAHSEE failure 
affects the probability of transferring to another district within the state, because students will 
be subject to the CAHSEE requirement in any district within the state. 

Finally, we estimate the effect of failingthe CAHSEE on the probability of graduating 
from the district, using a binary indicator of graduation status provided by the districts. For 
both the persistence and graduation variables, we have data only for the 2005, 2006 and 2007 
cohorts, because our most recent data from the districts is from Fall 2007. For the achievement 
variable (CST scores), we have data for each of the 2005-2008 cohorts, because this variable is 
collected in 11 th grade. 



Findings/Results: 

Our preliminary results show that, conditional on 10 th grade test scores, there are no 
significant differences in the 11 th grade achievement between students in the pre-policy (2005) 
cohort and students in the policy cohorts (2006-2008). Moreover, there are no differences 
between cohorts for students in the bottom quartile of 10 th grade test scores. 

The same is true for persistence— conditional on 10 th grade test scores, there are no 
significant differences in the persistence to Spring of 12 th grade between students in the pre- 
policy (2005) cohort and students in the policy cohorts (2006-2007). This is true for students in 
the bottom quartile of 10 th grade test scores as well as in the upper quartiles. 

However, when we examine graduation rates, we find substantial differences in 
graduation rates, conditional on 10 th grade test scores, between students in the pre-policy 
(2005) cohort and students in the policy cohorts (2006-2007). These differences are 
concentrated among students in the bottom quartile of the 10 th grade test score distribution. 
For students in the bottom quartile, graduation rates among those in the pre-policy cohorts 
were 10-12 percentage points higher than those in the policy cohorts; for those in higher 
quartiles, graduation rates were similar between the cohorts. 

These results suggest that the implementation of the policy did not affect students' 
achievement or persistence to 12 th grade, but did substantially reduce graduation rates among 
low-achieving students. 

Conclusions: 

Most arguments in favor of high school exit exams imply that the exams will motivate 
students to work harder to pass the exit exams. If this were the case, we would expect to see 
higher 11 th grade achievement levels, conditional on 10 th grade test scores, among those 
cohorts subject to the exit exam requirement, particularly among those at the low end of the 
achievement distribution. But we observe no such pattern. 

A primary concern of those opposed to high school exit exams is that the exams will 
discourage students, particularly low-achieving students, who fear they may not pass the exam. 
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which will lead to higher dropout rates. If this were the case, we would expect to see lower 
persistence rates among low-achieving students who were subject to the policy than among 
those who were not. Again, however, we observe no such pattern. 

The pattern we observe, however, suggests that the exit exams do not meet up to the 
expectations of their advocates nor the fears of their opponents. Rather, low-achieving 
students required to pass exams do not learn any more than they would had they not been 
subject to the requirement; nor do they persist in school any less than they would have. The 
sole effect of the policy appears to be deny diplomas to 10% of low-achieving students who 
persist in school through 12 th grade. 



5 




Appendixes 



Appendix A. References 

Center on Education Policy. (2004). State High School Exit Exams: States Try Harder But Gaps 
Persist. Washington, DC: Center on Education Policy. 

Center on Education Policy. (2005). How have high school exit exams changed our schools? 

Some perspectives from Virginia and Maryland. . Washington, DC: Center on Education 
Policy. 

Dee, T. S., & Jacob, B. A. (2006). Do High School Exit Exams Influence Educational Attainment or 
Labor Market Performance? Unpublished manuscript. 

Martorell, P. (2005). Do high school exit exams matter? Evaluating the effects of high school 
exit exam performance on student outcomes. University of California, Berkeley. 

Papay, J. P., Murnane, R. J., & Wlllet, J. B. (2008). The conseguences of High School Exit 

Examinations for Struggling Low-Income Urban Students: Evidence from Massachusetts: 
National Bureau of Economic Research. 

Reardon, S. F., Arshan, N., Atteberry, A., & Kurlaender, M. (2008). High Stakes , No Effects: 

Effects of Failing the California High School Exit Exam. Paper presented at the Annual 
Meeting of the Association for Public Policy Analysis and Management. 

Reardon, S. F., & Galindo, C. (2002, April). Do High-Stakes Tests Affect Students' Decisions to 

Drop Out of School? Evidence from NELS. Paper presented at the Annual Meeting of the 
American Educational Research Association, New Orleans. 

Warren, J. R., Jenkins, K. N., & Kulick, R. B. (2006). High School Exit Examinations and State-Level 
Completion and GED Rates, 1972-2002. Educational Evaluation and Policy Analysis, 

28 ( 2 ). 

Wise, L. L., DeMeyer Harris, C., Sipes, D. E. S., Hoffman, G. R., & Ford, P. J. (2000). High School 
Exit Examination (HSEE): Year 1 Evaluation Report. Sacramento, CA: Human Resources 
Research Organization, for the California State Department of Education. 




